Whether termed computer vision, machine vision, or the like, systems providing automated object recognition have traditionally been the subject of considerable interest for implementation in industrial and military applications. One of the ongoing challenges encountered by any system providing object recognition is the variability in the images presented by the same object, and even by the same object viewed from the same perspective, as the result of environmental factors such as the changes in light and shading produced by weather patterns, seasonal transitions, and shifting daylight over the course of a single day. In order to address this and other significant technical challenges, most systems providing reliable object recognition utilize one or more complicated algorithms requiring substantial computing resources for their execution. As a result, many conventional approaches to achieving object recognition may provide identification of an object of interest only after a considerable delay, require a costly and sophisticated computing platform for the processing of object images, or may suffer from both disadvantages. Still, some effort has been directed to provide object recognition as discussed in U.S. patent application publication 2005/0208457 to Fink et al. titled “Digital Object Recognition Audio-Assistant for the Visually Impaired”, filed Jan. 5, 2005.
Unfortunately in light of the conventional state of the art, object recognition systems could provide particular benefits to individuals suffering from a sensory impairment, such as blind or visually impaired persons, for example. In principle, the sight of a visually impaired individual might be effectively enhanced by the object recognition capability of an electronic assistant type device providing that functionality. However, in order for an electronic device providing object recognition to most fully benefit the visually impaired individual, the device should address at least three criteria. One of those criteria is that it is desirable that the device providing electronic sight enhancement be portable, so that the visually impaired individual can readily transport the device as desired. One of those criteria is that it is desirable that the device provide a flexible and user friendly interface enabling the visually impaired user to activate and control the object recognition functionality of the device. Furthermore, in order to most fully empower the visually impaired individual while also protecting their safety, it is desirable that the device providing object recognition do so reliably, and do so in real-time, thus enabling the visually impaired user to engage features of his or her present environment.
However, as discussed above, the computing resources required by conventional approaches to achieving object recognition are often substantial. That computing power requirement may considerably exceed the resources of a single portable device under even relatively relaxed performance standards. At best portable devices provide minimal support for character recognition as discussed in U.S. Pat. No. 7,627,142 to Kurzweil et al. titled “Gesture Processing with Low Resolution Images with High Resolution Processing for Optical Character Recognition for a Reading Machine”, filed Apr. 1, 2005. When the personal safety of the user and the efficacy of the system providing the object recognition functionality demand that identification of objects be highly reliable and be provided in real-time, the required computing resources far outstrip those available from a single portable electronic device.
One possible avenue for aiding visually impaired users includes interacting with a remote assistant. Ideally a remote assistant should be able to interact with the visually impaired user's full environment. Others have put forth effort toward enabling interactions between a visually impaired user and a remote assistant. U.S. Pat. No. 7,864,991 to Espenlaub et al. titled “System and Method for Assisting a Visually Impaired Individual”, filed Apr. 6, 2007, discusses visually impaired individual wirelessly sending audiovisual information about a situation to an assistant. The assistant can then reply back with a solution to the situation. Unfortunately, the assistant lacks complete a complete view of the individual's surrounding environment.
Another example includes U.S. patent application publication 2008/0043934 to Gallick titled “Communications Device for Visually Impaired Persons”, filed Aug. 4, 2006, which discusses providing a device having surface sensors to the visually impaired person where a remote assistant can observer the person's interaction with the device.
A more advanced effort includes U.S. patent application publication 2008/0198222 to Gowda titled “System and Method for Tele-presence”, filed Feb. 4, 2008, which takes the concept of remote assistance a little further. Gowda indicates that a visually impaired subject can be connected with a guide where the guide can use multi-modal information about subject's environment to aid the subject, possibly by directing the subject to an object. Still, the assistant is limited to the view provided by the subject and also lacks the ability to acquire additional data about the environment.
Interestingly, at best, only limited effort has been directed to utilizing augmented reality systems to aid visually impaired individuals. What has yet to be appreciated is a remote assistant can interact with an augmented reality panorama representing a visually impaired person's environment and its objects. The augmented reality panorama can be built based on sensor data collected via the individual's smart phone, or other sources of ambient data. The assistant can interact with objects, or other aspects augmented reality panorama, and send device commands back to the individual's smart phone to assist the individual or gain further informant about the environment.
Unless the context dictates the contrary, all ranges set forth herein should be interpreted as being inclusive of their endpoints, and open-ended ranges should be interpreted to include commercially practical values. Similarly, all lists of values should be considered as inclusive of intermediate values unless the context indicates the contrary.
Thus, there is still a need for remote assistance for visually impaired individuals.